Software development usually requires testers to evaluate the quality of a software product in many ways, including any content such as images incorporated in a software build or generated by execution of a software build. Traditional testing of such images may involve verifying that every pixel of a test candidate image is identical to the corresponding pixel of a master image. A typical validation algorithm may directly compare each pixel of a test candidate image to a corresponding pixel in the master image and then decide to pass or fail the test candidate image based upon the number of direct pixel comparison failures. The problem with this approach is that the comparison may fail from numerous slight differences that may not be visible by human perception or not relevant with respect to the definition of correctness for the comparison.
Another problem with this approach is that it does not account for noise that may be introduced into an image such as by disk drive controllers in accessing the image from persistent storage or by rendering drivers in rendering an image retrieved from storage. Based on the specifics of the system, some other common sources of noise may include rounding errors due to the high precision of the computations, differences in bit representation of the image format, visually detectable artifacts such as salt and pepper noise from spurious pixels, information missing from the image or extraneous information added to the image, and so forth. Additionally, differences in the implementations of graphics hardware and software drivers, including renderers, may create differences in two rendered images. In any of these cases, a direct pixel comparison approach may often fail where noise has been introduced in the image. Moreover, a direct pixel comparison may not allow for selective targeting of part of an image for validation such as a graphical component like a button or control.
What is needed is a more flexible and adjustable system for validating images. Such a system should be able to compensate for noise introduced into an image, allow for adjustable control of the resolution of an image and allow for selective targeting of content for validation based on a subset of the initial properties if needed.